Hi,
I've programmed a DXL-script which exports all modules in a folder to a excel-sheet. The problem is, that the execution time of the export is getting slowly the longer the script is running. Has anybody an idea which problem the script have? I've attached the script. The script is not complete, but you see the important part in which I suspect the problem.
Thanks! tobi_mehrl - Tue Aug 04 04:52:49 EDT 2015 |
Re: Script is getting slowly the longer the script is running The problem you are experiencing is very common in DXL. There are to resource problems that will make your script become slower and slower: - Allocation Leaks: You allocate something and do not deallocate it after use (e.g. a Buffer, Skip, OleAutoObj, ...) . Allocated objects pile up. DOORS manages a list of allocated objects (in a very inefficient way), so all further allocations will be slowed dramatically (up to factor 100x slower) when you reach a certain amount of allocated objects.
- String Table littering: You create a (real) lot of strings or a lot of large strings (we are talking millions of strings). This is also a very common with OLE exports, since the DOORS OLE Api does not allow you to pass a Buffer and will internally persist every string that goes out to an ole method as parameter in the string table. When a lot of clashes exist inside the string table, the check if a string is already contained in the string table becomes slower.
In your case most likely the allocations are the problem - there are numerous posts about this issue and I also posted a script that will help you detect these kinds of problems and verify that you resolved them. When you try to do VERY large exports, then you might also hit the string table littering problem. This one might not be resolvable as long as you use OLE and pass a lot of data over it. I had the same problem with an XML export which lead me to writing my own XML library instead of using a COM object. So: Search the forum for "allocation" "leak" "memory" or something like that and you will find the information you need to resolve your problems. By the way - the attachment did not make it in your post. Regards, Mathias |
Re: Script is getting slowly the longer the script is running Hi, I thought as much that the problem is feeded by allocationy problem. In point of fact I use buffers. I have read the dxl reference for a buffer function like "clear" or a function in this way, but I don't find a function which reclaims memory?! Now, you get the dxl-file. Sorry. Attachments output.dxl |
Re: Script is getting slowly the longer the script is running tobi_mehrl - Tue Aug 04 05:24:40 EDT 2015 Hi, I thought as much that the problem is feeded by allocationy problem. In point of fact I use buffers. I have read the dxl reference for a buffer function like "clear" or a function in this way, but I don't find a function which reclaims memory?! Now, you get the dxl-file. Sorry. I am pretty sure, that the problem is located inside the "setCell" method ... You can test, by commenting out the setCell calls, which should make your script run without slow down. As for the two buffers, you are already freeing them, so that seems fine ( delete(Buffer) ) Note that the allocation Problem also applies to OleAutoObj, which means, that EVERY ole handle that you aquire with oleCreateAutoObject, oleGetAutoObject, oleMethod, oleGet ... must later be freed with oleCloseAutoObject. That usually adds a lot of overhead to OLE calls, when your library is build the wrong way. Unfortunately most OLE libraries that you find (Excel.inc, etc.) do not take into account the closing of the handles, therefore they are doomed to slow your program down. Hope this helps, regards, Mathias
|
Re: Script is getting slowly the longer the script is running Thanks for your answer. I have another question. In a another function in my script the function "fullAttributeValue_()" is used, but this function is not declared in my script. At the beginning of the script, there are 2 files included: ole.inc and itfutil.inc. I have opened this two files but in no of this two files was found this function "fullAttributeValue_()". Where is this function defined? Do you know this function? |
Re: Script is getting slowly the longer the script is running tobi_mehrl - Tue Aug 04 08:30:35 EDT 2015 Thanks for your answer. I have another question. In a another function in my script the function "fullAttributeValue_()" is used, but this function is not declared in my script. At the beginning of the script, there are 2 files included: ole.inc and itfutil.inc. I have opened this two files but in no of this two files was found this function "fullAttributeValue_()". Where is this function defined? Do you know this function? Hi > fullAttributeValue_() IBM\DOORS\9.3\lib\dxl\utils\attrutil.inc Best regards Wolfgang |